Performance of distributed multi-agent multi-state reinforcement spectrum management using different exploration schemes

0957-4174/$ see front matter 2013 Elsevier Ltd. A http://dx.doi.org/10.1016/j.eswa.2013.01.035 ⇑ Corresponding author. Tel.: +1 514 577 9759. E-mail addresses: [email protected] (A.H.R. K (R. Sabourin), [email protected] (F. Gagnon). This paper introduces a novel multi-agent multi-state reinforcement learning exploration scheme for dynamic spectrum access and dynamic spectrum sharing in wireless communications. With the multiagent multi-state reinforcement learning, cognitive radios can decide the best channels to use in order to maximize spectral efficiency in a distributed way. However, we argue that the performance of spectrum management, including both dynamic spectrum access and dynamic spectrum sharing, will largely depend on different reinforcement learning exploration schemes, and we believe that the traditional multi-agent multi-state reinforcement learning exploration schemes may not be adequate in the context of spectrum management. We then propose a novel reinforcement learning exploration scheme and show that we can improve the performance of multi-agent multi-state reinforcement learning based spectrummanagement by using the proposed reinforcement learning exploration scheme. We also investigate various real-world scenarios, and confirm the validity of the proposed method. 2013 Elsevier Ltd. All rights reserved.